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ABSTRACT 



This paper describes the attempt to use the Internet to 
study a particular student culture. The goal is to highlight the ethical and 
procedural issues encountered in the Internet study of an online student 
culture. The primary purpose of the study was to examine how closeted gay 
students experience their identity on campus and in an online environment. 
Other than the invitation to participate, no messages were sent to chat rooms 
and no conversations or discussions were initiated in the public forums. 
Researchers only responded to inquiries that were directed to them privately, 
and were careful to follow the community guidelines for chat rooms they 
entered. Ultimately 13 individuals consented to participate in the study, and 
7 agreed to be interviewed online. In addition to the formal data collection, 
researchers engaged in more than 100 exchanges with potential participants. 
The research application was readily accepted by the college's institutional 
review board. Several technical problems were evident, especially with regard 
to Internet access. A positive aspect of the research was that costs for data 
collection were minimal. The greatest benefit of the approach was that the 
participants' anonymity was virtually assured. It also appears that the 
anonymous culture of the Internet caused many dropouts and no-shows among 
those the researchers expected to participate. The researchers thought that 
they were not altogether successful in negotiating the Internet culture, and 
they recognized some problems with reliability and validity. They did attract 
respondents from 11 different states and at least that many institutions, and 
they thought the study made a real contribution to knowledge about closeted 
gay students on campus. (Contains 12 references.) (SLD) 
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Studying Student Culture via the Internet 

This paper describes our attempt to use the internet to study a particular student culture. 
The findings of the study itself, reported at an earlier conference (Mueller, Brownell, & Kinser, 
2000) and in a separate paper in preparation, are here of secondary importance to the 
methodology we used. Our primary goal is to highlight the issues - ethical and procedural - 
encountered in conducting an internet-based study of an on-line student culture. First we briefly 
discuss the idea of an on-line student culture and our rationale for a chat room study of one such 
population. Next we turn to the study we conducted and the practical issues that arose as we 
collected our data. Finally, we conclude with our recommendations for other researchers who are 
considering a similar path. 

On-line Student Culture 

Students at most colleges and universities now have relatively convenient access to fast 
network connections, and they use the internet for regular communication with peers both on 
campus and off (Gatz and Hirt, 2000). Since computers have always been a part of the lives of 
today’s college students (a traditional first-year student would have been bom in 1981, the year 
IBM released its first PC), they are generally familiar and comfortable with the technology 
involved. Likewise, the World Wide Web has an intimate place in their history — current 
undergraduates were in grade school when the first graphical browsers were developed, and by 
the time they got to high school, web browsing was the most popular internet activity. 

As the internet increasingly pervades undergraduate student life, it becomes important to 
consider the role that on-line communities play in the development of a student culture. These 
communities, developing as often around common interests that cross campus lines as they might 
around more proximate issues, have been recognized as playing a significant role in the lives of 
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the general internet-using public (Kendall, 1999; Liu, 1999). For faculty, email and listservs have 
transformed scholarly communication (Bennett, 1997), and on-line discussions {The Chronicle of 
Higher Education's “Colloquy” for example) have become a common means of academic debate 
and discussion. Students have certainly not been immune to similar influences: web pages, 
listservs, chat rooms, and instant messaging can be seen as simply an updated version of the late 
night dorm room “bull session.” The students involved may or may not be on the same campus. 
Nevertheless, a group can potentially form which enjoys shared values, behavior patterns, ideals, 
artifacts, language, and ways of viewing the world (Liu, 1999), arguably meeting the definition 
of a student culture (Kuh & Whitt, 1988). 

Our decision to study an on-line student culture was driven by the requirements of the 
population we wished to understand: closeted gay college students. The existence of a gay 
student culture has been documented (e.g., Rhoads, 1994), and several theories of gay identity 
development have been proposed (Evans, Forney, & Guido-DiBrito, 1998). Much of this 
research and theory, however, is based on studies of students who are openly gay. This point 
should not be discounted. To know about the closeted student experience primarily through the 
experience of students who have come out of the closet is problematic. As Rhoads (1994) 
documents, coming out — the process of moving from being in the closet to publicly 
acknowledging one’s same-sex attractions - is intricately connected to gay student culture. An 
important part of our understanding of that culture is lost if we do not specifically seek out those 
students who are in the earliest stages of the coming out process. 

Finding and talking to these students on a college campus is easier said than done, 
however - by definition, closeted students intentionally hide their identities. Long-term 
ethnographic research is effective, but practical only for single institution case studies. We 




4 



3 



Studying Student Culture 
Draft 

wanted to find a way to communicate with students from a number of different institutions who 
were still closeted without devoting years of our lives to collecting data. Our solution was to use 
gay-oriented chat rooms on the internet 1 . Because of the anonymity these sites provided, we 
presumed that closeted students would be among the participants in the chat room discussions. 
And because the chat rooms were not limited to a particular campus, we believed that closeted 
students from a variety of institutions would be represented. We decided, then, to study the 
closeted gay student culture as it was exhibited in internet chat rooms and compare that to what 
we know from other, more traditional studies of gay student culture. 

The Study 

The primary purpose of our study was to examine how closeted gay students experience 
their identity on campus and in an on-line environment. None of us had experience in internet- 
based research, however, and we could readily see the ethical and procedural issues involved in 
using that approach (Frankel & Siang, 1999). Therefore, our study was designed secondarily to 
explore the use of the internet for conducting research on student populations in higher 
education. As stated earlier, the findings related to the primary purpose have been reported 
elsewhere. This paper focuses on our findings related to internet-based research. 

We recruited participants through chat rooms devoted to gay issues and college life on 
America On-Line and Gay. Com. Jayne Brownell and John Mueller, using the screen names of 
LGBResearch and ResearchLGB, posted messages inviting closeted gay students to participate 
in our study. We maintained profiles for each screen name which identified the researcher, his or 
her institutional affiliation, and complete contact information; however, we declined to offer 
personal statements or descriptive information (other than gender) as part of the profile. Each 

1 We are indebted to a master’s student who first suggested this idea to us in a class on student cultures. He was not 
interested in pursuing this line of research, however, and declined our invitations to join us in conducting the study. 
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profile also included a link to a web page that we designed to provide more detail about the 
study. 

Other than the invitation to participate, we posted no other messages to the chat rooms 
and no conversations or discussions were initiated in the public forums. Both the messages and 
our profiles directed interested individuals to a web site for more information about our project, 
and we ignored all comments about the study that were made in the chat room discussions. And, 
even though we could have identified many potential participants through reading the chat room 
discussions and profiles, we did not contact any individuals until we were contacted directly. We 
only responded to inquiries that were directed to us privately, either through a private chat 

message or through regular email. 

We verified th< ; our procedures would not violate the official community guidelines (the 
list of rules that governed discussion) for the chat rooms we entered. Towards the end of our data 
collection, the guidelines for the Gay.com sites changed to specifically state that research was 
prohibited. Since we were using the space only to recruit subjects, we did not believe we were in 
violation of that rule. We nevertheless attempted to get explicit permission from the chat room 
managers, informing them that we would only be posting messages which we felt stayed within 
the spirit of the community guidelines. We received no response to our emails requesting 
permission, so we discontinued our posts in those chat rooms that had the research restriction. 

The postings yielded 38 formal inquiries, and numerous informal questions. We emailed 
a consent form and a survey to those who indicated interest in our study. Ultimately 13 
individuals consented to participate in our study, seven of whom also agreed to be interviewed 
for the study. These interviews took place on-line in a private chat space. We did not attempt to 
discover any additional information about the participants other than the information we 
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explicitly requested in the survey and the interview. It would have been easy to lurk in chat room 
discussions, peruse participant profiles, or search internet archives for more information. We 

decided, however, not to be surreptitious in our data collection. 

- q n addition to our formal data collection; we engaged in over 1 00 exchanges, also in 

.; ••• private chat rooms, with potential participants who may or may not have been closeted college 
students. For any number of possible reasons, most of these individuals decided not to complete 
the consent form or the survey. For those who indicated that they were closeted only during an 
informal chat (i.e., before completing a consent form), we did not use their campus experiences 
for the study. In this paper, however, the impressions we formed of these conversations are used 
to help us understand the dynamics of conducting i search with this population in a chat room 
environment. For the sake of simplicity, we use the term “participant” to refer to any individual 
with whom we had a conversation, regardless of whether they formally participated in the study 
or only had a dialogue with us regarding their potential participation. 

Issues Encountered 

While the study was largely successful and helped us gain some interesting insight into 
the experiences of closeted students, several issues, as expected, did affect the research process. 
Institutional review 

Approval from a research institutional review board (IRB) involved not only the typical 
methodological descriptions, but also an explanation of the technology to be used and, given the 
anonymous nature of the medium, how the study could be completed in an ethical manner. 
Because we expected close scrutiny of our decisions, we paid very close attention to these issues 
in preparing the review protocol. We highlighted the potential risks to the participants, 
emphasizing the need to protect their identities. We described the chat room technology and how 

ERIC 
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private chats could be conducted in real-time outside the view of other chat room participants. 

We noted the lack of guidelines in conducting this sort of research, and the assumptions we made 
in constructing our own. We described our.process for obtaining informed consent, the necessity 
of engaging in conversations with potential participants before obtaining consent, and our 
procedures for ensuring that no data would be used in the final analysis without consent. 

Surprisingly, our application was accepted through expedited procedures without a full- 
review - it was approved by a single member of the IRB who determined our study did not pose 
a risk to the participants. This decision was, frankly, somewhat unsettling given the novel way 
we proposed reaching a rather sensitive population. We would like to believe that our obsessive 
concern with ethics satisfied all concerns and smoothed the way for a quick approval. It is 
possible, however, that because two aspects of our study (the chat room technology, and the 
closeted gay student culture) may have been foreign to the initial reviewer, the full significance 
of the risk to the participants may have been lost in the dry language of the IRB form. Secondly, 
we must note that we are from three different institutions: Louisiana State University (Kevin 
Kinser), Indiana University of Pennsylvania (John Mueller), and Columbia University (Jayne 
Brownell). The institutional affiliation of John Mueller, the lead researcher for this study, 
dictated where we submitted our IRB application. A submission to Columbia or Louisiana State 

universities might have resulted in a different assessment. 

While the outcome was appreciated in the sense that it allowed us to move forward with 
our study, it did not provide us with the institutional endorsement we were anticipating. We 
believe our use of unfamiliar technology played a role in making the IRB approval process less 
useful to us than it could have been. 
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Technical considerations 

Several technical problems were evident. We experienced trouble connecting to the 
internet during times scheduled for an interview. On other occasions, for no discernable reason, 
we would not be able to enter a chat room. Long pauses between typed messages were common, 
and it was hard to tell if that meant the participant was thinking, was composing a long response, 
was engaged in another chat, or had simply got skittish and decided to end his or her 
participation. Typed messages tended to lose the nuances of face-to-face interviews. The bottom 
line was that many of the skills we had learned to put interviewees at ease were unavailable to us 
because of our own technological nervousness. The follow-up questions and probes that would 
be so natural to use in response to non-verbal cues, for example, had to find a more formal place 
in our interview protocol. We, as the researchers, were generally not as comfortable with the 
format as were the participants in the study. Eventually, we were able to adapt our interviewing 
style to the on-line communication process, but it was very different from the easy level of 
conversation that typically marked our face-to-face interview experiences. 

On the positive side, costs for data collection were minimal at the same time that 
geographic barriers were eliminated. Interview transcription was simplified, though not always 
automatic. America On-Line, for example, had a “save chat” function that simply copied our 
discussions to a file. When that device was not available, our other option was to cut and paste 
portions of each exchange into a separate document. While this latter option was less time 
consuming than typical transcription from a tape, it was distracting to continually switch back 
and forth between documents while the interview was taking place. The biggest benefit that we 
gained from the technology, however, was the fact that the participants’ anonymity was virtually 
assured. No real names, mailing addresses, or phone numbers were needed, and since we used a 
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chat room environment, misdirected email that could be opened by the wrong user was not a 
significant concern (Frankel & Siang, 1999). Anonymity is what attracted us to the medium in 
the first place, and we were able to take advantage of it for our study. 

Confidentiality and trust 

- ... .These two issues went hand in hand. As students whose gay identities were not known to 
others, we had a particular responsibility to ensure confidentiality. Participants had to place an 
inordinate amount of trust in us based primarily on typed assurances. We addressed this issue by 
being clear about who we were in any on-line exchange and through our on-line profiles which 
were available for anyone to see. We shared our names and institutional affiliations and used our 
university email addresses so that students could verify our identity. We let everyone who 
contacted us know that we planned to write about this for our colleagues. When asked, we were 
clear about our sexual orientation and why we were interested in this topic, both personally and 
professionally. 

Trust did not come easily for any of the participants. We had to build it slowly. No one 
agreed to participate immediately. Most wanted to have a conversation about our motives, 
particularly when they found out only one of us identifies as gay. These conversations occurred 
before the students formally agreed to participate in the study, and required us to devote 
substantial amounts of time with potential participants who would never end up joining our 
study. For those who did decide to participate, however, this extensive dialogue seemed 
necessary to help them become comfortable with our integrity. 

We do know that the risk of identification did cause several students to decline 
participation in the study, particularly those whose only email address was in the .edu domain. 
Even though we stated that we would not place their campus identities at risk, many potential 
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participants were understandably nervous. Even though identity is easily masked in a chat room 
environment, students still perceived that, by volunteering information, an unscrupulous person 
(one of us, presumably) would link them to their home campus and place their straight identity in 
danger. For .these students our assurances - not to mention our consent form and IRB approval - 
apparently held little stock. 

Internet culture considerations 

Much as the culture of a particular campus influences the development of student culture, 
the overall culture of the internet has an impact on the expectations and mores of on-line 
communities (Parks & Roberts, 1998; Porter, 1997). As outsiders, we were perceived by some to 
be violating the unstated norms of the on-line communities we entered. In several ways, this, 
perception was correct. We presented ourselves as researchers and faculty members and not 
: fellow gay students. We were offering not the typical chat room conversation, but a focused 
dialogue on the participants’ experiences as a closeted gay or lesbian on a college campus. And 
we stated openly that we intended to publicize our findings rather than maintaining the world- 
apart nature of many internet-based communities (Fernback, 1999). 

Our initial post to a chat room would often result in brief conversations among the 
participants as to whether our request was appropriate. This was particularly evident when John 
Mueller, as a male, would post to lesbian chat rooms. The perception that he was invading a 
presumptively female space was made clear on several occasions through the ensuing dialogue, 
and, in one room, the administrator told him he had to leave. This was less a factor for Jayne 
Brownell in the gay male rooms - the participants more frequently commented upon her gender 
with pleasant surprise than with suspicion or hostility. On the other hand, a gender match 
between the researcher and the population in the chat room frequently resulted in individuals 
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responding with false interest in the topic. Messages would be sent to us which would begin by 
asking about our study, then move to queries about our age and height and other personal 
information. While these were common enough questions for the chat rooms, we refused to 
reveal such detail about ourselves, and those who approached us with such personal questions 
were never seriously inclined to participate in our study. 

We also believe that the anonymous culture of the internet caused us to have many drop- 
outs and no-shows among those who we thought had made a commitment to participate. Either 
they would request a survey and consent form and never return it, or they would agree to an 
interview, and then not be on-line at the appointed time. One participant in particular broke three 
separate appointments for an interview before we finally dropped him from consideration. 

. Finally, participants would often maintain conversations with others in the chat room at 
the same time we were attempting to talk with them about our study in a private space. We had 
assumed that we would receive the participant’s undivided attention. Apparently, however, 
maintaining multiple conversations was part of the culture in many of the chat rooms. Our 
expectation that individuals chat only with us was not always shared by the participants. In fact, 
on several occasions the participant was surprised to find out that one could chat in a private 
room exclusively without maintaining a presence iii the public chat space. While we tried to 
insist that our conversations be one-on-one, we had no way to enforce this restriction and it was 
breached on occasion. In these instances we were usually able to quickly re-establish the priority 
of the interview by simply asking the participant to discontinue his or her other chats for the 
duration of our discussion. 

Frankly, we were not altogether successful in negotiating the internet culture issue. From 
our gender-based infractions to violating the conventions of multi-person dialogue, our venture 
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into this domain was more obtrusive than we had naively hoped. We did not spend as much time 
as we should have in learning about the culture before attempting to enter it. And once in, we did 
not always attend to the impact our mere presence could have on the chat room members. We 
humbly admit our errors and ignorance in this respect and hope that by relating our mistakes, 
others will avoid them. 

Reliability and validity 

Reliability and validity are concerns with any research, all the more so on the internet 
where identity is so indeterminate. Were the students we interviewed and surveyed actually gay? 
Were they truly closeted? Were they even students? This is, after all, one of the benefits of being 
on-line: you can be whoever you want to be. However, since it is practically impossible to ensure 
accurate identity, in such an environment, we chose to accept most participants at face value. 

There was one case wl:?re we felt we were receiving responses inconsistent with the individual’s 
stated identity, so we dropped that person from the study. Other than that, everyone with whom 
we had a substantive conversation seemed to be giving us an honest account of his or her true 
persona. Because all of the participants invested a substantial amount of time in our project, we 
doubt that many individuals would attempt to deceive us to that extent. As a joke, it is not that 
funny; and as a political statement, it lacks an immediately apparent audience. Parenthetically, 
we note that survey research which is not conducted on-line faces similar issues with respect to 
truthfulness (Stage, 1992). Simple tests for outliers can eliminate the more obviously problematic 
responses, but little can be done to guard against more subtle dissembling. To questions of 
identity, therefore, we chose to remain agnostic. 

Particularly with the population we wanted to study - closeted gay students - another 
question becomes central. They do not tell the truth about themselves to people they see every 
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day; why would they be honest with researchers they have never met? We decided not to attempt 
to verify the accuracy of the responses our participants gave us through any external or 
independent means. As we indicated above, we did not double-check their comments with 
information they may have in their profiles, nor did we lurk in a chat room to see if their public 
dialogues were consistent with the private chats they had with us. We also did not attempt to 
; assess the accuracy with which they described their campus climate, even though for most 
participants it was easy to guess what college they were attending. Again, we relied on the trust 
we had built with the participants through our conversations, and the amount of time they had 
committed to our study, to guard against prevarication and outright lies. In addition, almost all of 
our participants expressed a strong desire to help us add to the literature on gay student issues. It 
seems unlikely that m* ty would intentionally try to thwart their expressed purpose for 
participating by feeding us false information about themselves or their experience. 

Concerns regarding selection bias and unknown response rates, however, were a serious 
issue. The generalizability of our results was most certainly affected, and we struggled with how 
to represent the findings we did have. Our method was like printing a classified ad seeking 
subjects for a research study. We could not say how many people saw the ad and decided not to 
respond. We do know, however, that self-selection skewed our sample. Our participants were not 
representative members of the closeted gay student population we were trying to study. They 
were all very certain they were gay. Many were on the verge of coming out to their friends on 
campus. Most had already told at least one other person that they were gay. They generally were 
very eager to talk about their experiences, and they approached the topic from a rather 
intellectual or conceptual perspective. Clearly, these were students who had been giving this a lot 
of thought before we arrived, and they leapt at the chance to talk in depth about an issue that 
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concerned them. 

Nevertheless, we did attract a group of students to our study from eleven different states 
and at least as many different institutions. For that reason, the technology served its purpose. It 
allowed us to find participants who would have been inaccessible to us through traditional 

'i 

means. The fact that they were not a statistically valid sample was beside the point in comparison 
to the wealth of information they provided. Without making claims of representativeness, we 
believe the real voices in our study are an important addition to what we know about being 
closeted on a college campus. 

Recommendations 

With the increased use of telecommunications technology in higher education, 
opportunities to use the internet as a research tool have become more obvious and practical. 
Techniques such as email surveys, real-time or asynchronous interviews, and web-based 
simulations have the potential to be an important part of the data collection repertoire for 
researchers of the college student experience. And, for populations (like closeted gay students) 
which would be difficult, if not impossible, to study via traditional means, the internet will prove 
to be an invaluable tool. 

Our initial foray into this realm generated findings that we believe add to the student 
culture literature. It was not without stumbles, as noted above. From our experience, we make 
several recommendations to other researchers considering similar projects. 

• View the institutional review as an important step in the research design process. It is an 
opportunity not only to educate members of the review panel about the technology 
involved, but also to identify and clarify technological limits as well. Do not proselytize 
the committee. Rather, work with it to develop a realistic model that maintains 
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appropriate ethical standards within the larger tradition of social science research. 

• Negotiate entry into an on-line community carefully. One poorly worded message or 
overly presumptuous request can quickly close off access to a potential subject pool. Be 
familiar with community guidelines and terms of sendee, both as they are written and as 

. .. .. they are actually enforced. The well-documented notions of “getting in” and “getting 
along” are equally applicable in the world of the internet as in traditional ethnographic 
research. 

• Recognize and respect the similarities and differences between the real world and the 
virtual (Parks & Roberts, 1998 ). Students often move back and forth with ease and our 
research suggests that the two environments axe linked in substantive ways. At the same 
time, there is a personal freedom inherent in the medium, particularly for the population 
that we studied, that is different from the constrictions imposed by campus-based 
interactions. For example, expect that personal questions and blunt criticisms will be 
common responses to requests for participants - reactions that are less typically found in 
traditional research. On the other hand, one’s real-life identity and institutional affiliation 
helps to provide the grounding on which trust can be built in the virtual environment. 
Student flexibility regarding the reality of the virtual environment should be appreciated 
and not resisted. 

• Do not worry so much about verifying the identity of the respondents; it is nearly 
impossible to do so with certainty. If the data look consistent, make the honest argument 
for validity. Do, however, be concerned about representative samples. Randomness is 
difficult to achieve and, in most forums, response rates cannot be determined. For these 
reasons, broad generalizations should be made with caution. 



O 

ERIC 



16 



15 



Studying Student Culture 
Draft 

• Be upfront about the intent of the study and make it easy for participants to opt out. This 
would involve a personal commitment not to use unobtrusive observations (e.g., lurking, 
transcripting chat room dialogue, etc.) in the study without permission. More specifically, 
informed consent should be rigorously enforced (Frankel & Siang, 1999). The transient 
nature of on-line populations means that silence cannot equal acquiescence. 

• Be prepared for technical glitches and data limitations. The server will go down just as 
you are beginning an interview. Email messages will get lost. Download times will slow 
to a crawl. And despite the liberal use of emoticons - typing :-( to convey sadness, for 
example - the affective content of messages will be difficult to discern. 

• Decide how to handle chat room discussions, concerns, and questions. The internet is, as 
its name suggests, an interactive medium. This means that chat room members will feel 
quite comfortable asking questions in response to research inquiries, whether or not they 
are participating in the actual study. This can be time consuming if decisions are not 
made in advance as to how to handle the replies. We suggest a web site devoted to 
frequently asked questions, and form letter responses as appropriate. In addition, whether 
or not to engage in public discussion in the chat room with potential participants is an 
important part of the research design. It should not be dependent on the particular 
discussion topic or researcher whim, but rather on the data collection requirements of the 
study. 

Conclusion 

We do not pretend to be experts on internet-based research. However, we do feel that the 
literature on this topic is limited, and that our contribution here can prove useful to other 
researchers. From distance learning to scholarly collaborations, the internet and associated 
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technologies are making a significant impact on the academy. Researchers of the student 
experience should be prepared to develop and use new methods, as well as to recognize and 
explore the role of the internet in campus life. 
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